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DETAILED ACTION 



Specification 



1 . Claim13 is objected to because of the following informalities: The limitation "an 
ID for identifying the synthetic key frame" is given twice in the claim. Appropriate 
correction is required. 



2. The following Is a quotation of the second paragraph of 35 U.S.C. 112: 

The specification shall conclude with one or more claims particularly pointing out and distinctly 
claiming the subject matter which the applicant regards as his invention. 

3. Claims 1-3, 5-7, 10-18, 24, 28, and 29 are rejected under 35 U.S.C. 112, second 
paragraph, as being indefinite for failing to particularly point out and distinctly claim the 
subject matter which applicant regards as the invention. 

4. Claim 1 indefinitely uses "it" to describe, "said video stream." 

5. Claim 1 recites the limitation " selecting key frame(s) or key region(s) 
representative of a divided section " in the second paragraph. There is insufficient 
antecedent basis for this limitation in the claim. "Divided section" should be replaced 
with "one of the divided sections". 

6. Claims 1, 3, 6, 7, 12, 14, 17, 18, 24, 28, and 29 recite "or" which is vague 
language and should be replaced with "selections are one of these". 

7. The term "meaningful" in claims 1 , 2, 5, 1 1 , 1 2, 1 6, 1 8, 24, and 28 is a relative 
term, which renders the claim indefinite. The term "meaningful" is not defined by the 
claim, the specification does not provide a standard for ascertaining the requisite 



Claim Rejections - 35 USC §112 
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degree, and one of ordinary skill in the art would not be reasonably apprised of the 
scope of the invention. 

8. Claim 2 rejected as being indefinite because there are separate receiving and 
dividing steps, where the dividing step now is receiving. 

9. Claims 6, 10, 13, and 15 give a "can statement" in which several claim limitations 
are followed by a "can statement" which then provides more limitations that indefinitely 
further limit the claim. 



Claim Rejections - 35 USC § 102 

10. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 

form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1 ) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an International application filed under the treaty defined in section 
351 (a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

11. Claims 1-7, 9, 11, 12, 14-18, and 20-27 are rejected under 35 U.S.C. 102(e) as 
being anticipated by Hirai et al., Patent # 6,526,215, hereinafter Hirai. With regard to 
claim 1 , which teaches a method of generating key frames comprising the steps of: 
receiving a video stream and dividing it into meaningful sections, selecting the key 
frames, and combining key frames to form one key frame, Hirai teaches, in column 4, 
line 45 an apparatus for receiving moving picture data, dividing it into scenes, further 
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dividing it up Into cuts, and forming still images as representatives (key frames) of these 
subdivisions. 

12. With regard to claim 2, which teaches the dividing step including receiving video 
form a second source, Hirai teaches, in column 3, line 8 and in figure 5, receiving input 
from sources such as a movie, and a still image. 

13. With regard to claim 3, which teaches the selecting step of including key frames 
output from a second source, Hirai teaches, in column 3, line 8 and in figure 5, receiving 
input from sources such as a movie, and a still image. 

14. With regard to claim 4, which teaches that a section is a unit of a segment, Hirai 
teaches, in column 4 line 45, that a scene (section) comprises a plurality of cuts 
(segments). 

15. With regard to claim 5, which teaches a method of describing a key frame 
comprising, dividing a video into meaningful segments, and describing a list of key 
frames, Hirai teaches, in column 4, lines 50-52, a detecting means for designating said 
representative still images of said scenes and cuts to edit said moving picture. Hirai 
also teaches, in column 15, line 26, that video data belonging to the M-icon (key frame) 
is displayed in the form of a list. 

16. With regard to claim 6, which teaches the describing step including: an ID, a 
representative segment locator, a key frame list for identifying elements of a key frame, 
and can also include a fidelity value, and information on arrangement, Hirai teaches, in 
column 3, line 8, and in figures 4 and 5, video linked information including names, types, 
and lengths of media. Hirai further teaches, in column 15, line 26, video data belonging 
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to the M-icon that is displayed in the form of a list. Hirai further teaches, with regard to 
the items that can be included, in column 5, line 6, a degree of abstraction, and in 
column 9, lines 35-51 in conjunction with figures 8 and 10, Hirai teaches the 
arrangement of parent and child elements in a Hierarchy. 

17. With regard to claim 7, which teaches the information about the arrangement 
includes 2-D or 3-D location information, Hirai teaches, in column 19, line 50, that there 
is preparation of items in a hierarchical structure so that when information is not 
suitable, one of the items in a layer higher that the layer containing the information item 
can be used. 

1 8. With regard to claim 9, which teaches a key frame locator including: the location, 
an annotation, and a related segment, also a segment locator designates a segment 
represented by a corresponding keyframe, and a fidelity value, Hirai teaches, in column 
1 1 , line 9-15, information regarding address, name, and cut start/end points. Hirai also 
teaches, in column 5, line 6, that the degree of abstraction is determined for the images, 

1 9. With regard to claim 1 1 , which teaches an s-key frame including a key frame list, 
Ratakonda teaches, in column 15, line 26, that M-icons (keyframes) can contain a list 
of M-icons. With regard to claim 1 1 also teaching each element of a key frame list 
having a fidelity value, Ratakonda teaches in column 5, line 6, that the degree of 
abstraction is determined for the images. 

20. With regard to claim 12, which teaches a method of describing key frame data 
comprising: dividing a video into meaningful sections, and generating the keyframes. 
Hirai teaches, in column 4, line 45, an apparatus for receiving moving picture data, 



Application/Control Number: 09/800,999 Page 6 

Art Unit: 2173 

dividing It Into scenes, further dividing it up into cuts, and forming still images as 
representatives (key frames) of these subdivisions. 

21 . With regard to claim 1 4, which teaches that each element of a key frame list 
contains a key frame locator, Hiral teaches, in column 1 1 , lines 9-15, that layer and 
address information for each M-icon (key frame). 

22. With regard to claim 1 5, which teaches a key region locator including an ID, and 
a representative segment locator, that can include: a fidelity value, a annotation, and a 
list of related segments, Hirai teaches, in column 3, line 38 and in figures 4 and 5, 
source and individual frame names. Hirai also teaches a means of calculating a fidelity 
value (see column 5, line 6), an annotation (see column 1 1 , lines 9-15), and a list of 
related segments (see column 11, lines 9-15). 

23. With regard to claim 16, which teaches each element of a key region list 
including a fidelity value, Hirai teaches, in column 5, line 6, that the images contain a 
degree of abstraction. 

24. With regard to claim 17, which teaches information about arrangement includes 
either a 2-D or a 3-D locator, Hirai teaches, in column 19, line 50, the tree structured 
arrangement of the information has an advantage that the tag buttons can be easily 
searched. 

25. With regard to claim 18, which teaches a hierarchical video summary method 
comprising means of, dividing a video stream into meaningful sections, assigning key 
frames to a key image locator, a hierarchical summary list for describing lower summary 
structures, and structural information, Hirai teaches, in column 4, line 45, receiving 
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moving picture data, dividing it into scenes, further dividing it up into cuts, and forming 
still images as representatives (key frames) of these subdivisions. Hirai further teaches, 
in column 11, lines 9-15, information being contained in the elements of a hierarchical 
structure giving address information and video structural information. 

26. With regard to claim 20, which teaches that each hierarchical summary structure 
is represented by an image representative of a specific segment, Hirai teaches, in 
column 16, line 1 and in figure 1 , how in the hierarchy each M-icon (key frame) has its 
own information zone. 

27. With regard to claim 21 , which teaches that each component of the lower 
hierarchical summary list uses a hierarchical/recursive summary structure as a lower 
hierarchical summary structure, Hirai teaches, in column 9, line 7 and in figure 1, how 
the hierarchy is organized from the top level story to the next level scenes to the next 
level of cuts where a cut is a subset of a scene, and each of these M-icons (key frames) 
has a summary element. 

28. With regard to claim 22, which teaches that the hierarchical summary structure 
has summary level information, Hirai teaches, in column 10, lines 56-67, that each icon 
is given a layer level value (1,2,... from the bottom layer). 

29. With regard to claim 23, which teaches the hierarchical summary structure 
including a fidelity value, Hirai teaches, in column 5, line 6, and a level of degrees of 
abstraction as associated with frame images. 

30. With regard to claim 24, which teaches a method for providing a video browsing 
interface comprising: dividing a video stream into meaningful sections, and providing a 
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user interface to a predetermined display, Hirai teaches in column 4, line 45, an 
apparatus for taking in moving picture data and dividing it up into scenes and cuts. Hirai 
also teaches in column 4, line 60, a means for displaying said hierarchical structure. 

31 . With regard to claim 25, which teaches the user interface providing the synthetic 
key frame in the form of view, Hirai teaches in figure 1 , a visual representation of the 
icons (keyframes). 

32. With regard to claim 26, which teaches key frames being arranged in a time 
sequence, and the key frames arranged in a tree shape, Hirai teaches, in column 9, 
lines 35-51 and in conjunction with figures 8 and 10, how all of the M-icons are arranged 
in the order in which they occurred in the inputted moving picture, and that they are 
displayed in a hierarchical tree structure. 

33. With regard to claim 27, which teaches key frames assigned to each node in a 
TOC form, Hirai teaches, in column 9, lines 25-34 and in conjunction with figures 8 and 
10, managing information in an organized form, and then displays a TOC (figure for the 
hierarchy in figure 10. 

34. Claims 28 and 29 are rejected under 35 U.S.C. 102(e) as being anticipated by 
Ratakonda, Patent # 5,956,026. With regard to claim 28, which teaches dividing video 
into meaningful segments, Ratakonda teaches in column 2, lines 13-27 generating a 
summary of a video based on key frames by detecting shot boundaries, and locating 
shots. With regard to claim 28 also teaching providing a user interface, Ratakonda 
teaches, in column 13, line 35, providing a user interface. With regard to claim 28 
teaching selecting s-key frames according to the user input, and reproducing a segment 
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represented by the selected s-key frame, Ratakonda teaches, in column 5, lines 56-63, 
clicking on a particular frame and being able to display a normal playback of a video 
sequence. 

35. With regard to claim 29, which teaches a reproducing step that reproduces a 
segment related with constituent elements of the contents of the key frame, Ratakonda 
teaches in column 5, lines 56-63, clicking on a particular frame and being able to display 
a normal playback of a video sequence. 

Claim Rejections - 35 USC § 103 

36. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

37. Claims 8, 10, 13, and 19 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Hirai and Ratakonda. Hirai teaches a method of generating 
representative still images from a video sequence, a locating means comprising: a 
means of calculating a fidelity value (see column 5, line 6), a annotation (see column 

1 1 , lines 9-1 5), a list of related segments (see column 1 1 , lines 9-1 5), and information 
on arrangement (see column 11, lines 9-15), but doesn't teach elements in a keyframe 
list containing locators, the locator including an inherent ID, and image locator to locate 
the image, a representative segment locator, or a key image locator being a structure 
for designating an image using, a key image locator, a key frame locator, a s-key frame 
locator. Ratakonda teaches a video summarization system similar to that of Hirai, but 
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further teaches elements in a key frame list containing locators, the locator including an 
inherent ID, and image locator to locate the image, a representative segment locator, 
and a key image locator being a structure for designating an image using a, key image 
locator, a key frame locator, a s-key frame locator. With regard to claim 8, Ratakonda 
teaches, in column 6, lines 45-67, a means for locating items in a key frame list. It 
would have been obvious to one of ordinary skill in the art, having the teachings of Hirai 
and Ratakonda before him at the time the invention was made to modify the method of 
generating representative still images of a video sequence of Hirai to include the 
representative image locating means of Ratakonda. One would have been motivated to 
make such a combination because locating these key frames and the video segment 
they represent can help in editing and/or viewing the video sequence. 
38. With regard to claim 10, which teaches the locator including an inherent ID, an 
image locator to locate the region, a representative segment locator, a key region 
locator that can contain: a fidelity value, an annotation, and a list of related segments 
(all rejected supra). Ratakonda teaches, in column 6, line 60, a key frames having 
specific IDs. Ratakonda further teaches, in column 6, line 63, the region that a key 
frame represents. Ratakonda also teaches, in column 6, line 62, that the key frame kj 
represents a contiguous set of video frames. It would have been obvious to one of 
ordinary skill in the art, having the teachings of Hirai and Ratakonda before him at the 
time the invention was made to modify the method of generating still images of Hirai to 
include the keyframes having specific Ids, the region that a keyframe represents, and 
the key frame kj representing a contiguous set of video frames of Ratakonda. One 
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would have been motivated to make such a combination because the more information 
that is available, the easier it will be for the user to use the video software to effectively 
find the sought after segment, find out how representative it is, and find relative 
segments. 

39. With regard to claim 13, which teaches the locator including an inherent ID, and 
image locator to locate the region, a representative segment locator, a key region list, a 
fidelity value (as rejected supra), and information on arrangement (as rejected supra), 
Ratakonda teaches, in column 6, line 60, key frames having specific IDs. Ratakonda 
further teaches, in column 6, line 63, the region that a key frame represents. Ratakonda 
also teaches, in column 6, line 62, that the key frame kj represents a contiguous set of 
video frames. Ratakonda further teaches, in column 6, line 62, a list of the frames that 
a key frame represents. It would have been obvious to one of ordinary skill in the art, 
having the teachings of Hirai and Ratakonda before him at the time the invention was 
made to modify the method of generating still images of Hirai to include the key frames 
having specific Ids, the region that a key frame represents, that the key frame kj 
represents a contiguous set of video frames, and a list of the frames that a key frame 
represents. One would have been motivated to make such a combination because the 
more information that is available, the easier it will be for the user to use the video 
software to effectively find the sought after segment, find out how representative it is, 
and find relative segments. 
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40. With regard to claim 19, which teaches a key image locator being a structure for 
designating an image using: a key image locator, a key frame locator, and a s-key 
frame locator, Ratakonda teaches, in column 6, lines 45-67, a locator method for 
locating key images of all levels of the hierarchy, and all the frames which they 
represent. It would have been obvious to one of ordinary skill in the art, having the 
teachings of Hirai and Ratakonda before him at the time the invention was made to 
modify the method of generating still images of Hirai to include a locator method for 
locating key images of all levels of the hierarchy, and all the frames which they 
represent of Ratakonda. One would have been motivated to make such a combination 
because the inclusion of these location and identification elements will help the user in 
using the video software. 



41 . The prior art made of record on form PTO-892 and not relied upon is considered 
pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to 
consider these references fully when responding to this action. The documents cited 
therein teach a method of generating representative key frames from a video sequence. 
37. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Dennis G Bonshock whose telephone number is (703) 
305-4668. The examiner can normally be reached on Monday - Friday, 8:30 a.m. - 5:00 
p.m. 



Conclusion 
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42. If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (703) 308-31 16. The fax phone number 
for the organization where this application or proceeding is assigned is (703) 872-9306. 

43. Any inquiry of a general nature or relating to the status of this application or 
proceeding should be directed to the receptionist whose telephone number is (703) 305- 
3900. 




